About

Sannidhya (Sann) Malpani is a highly motivated big data developer with a passion for learning new technologies and solving complex problems. With 1.86 years of experience in the field, he is currently working as a Big Data Engineer at Incubyte, where he is responsible for moving a bank's on-prem infrastructure to the cloud. Sannidhya's expertise lies in big data, data engineering, SQL, Spark, Databricks, Python, Scala, Azure, Apache Airflow, CI/CD, DevOps, ETL, and infra. He has worked on multiple projects throughout his career, including refactoring and optimizing existing stored procedures on SQL server to reduce processing time by 2 hours. He has also refactored stored procedures into Databricks code to facilitate the move to the cloud and implemented CI/CD of Databricks notebook using Azure DevOps, adding tests and code coverage to enhance the quality of the code. Before joining Incubyte, Sannidhya worked as an Associate Big Data Engineer at Celebal Technologies, where he contributed towards solving the problem of getting loyalty points transactions in real-time. He gathered business requirements and effectively converted them into data pipelines, wrote multiple data pipelines from scratch in Azure Databricks, and implemented pipelines to gather data from various sources like Firebase, Branch.io, Freshdesk, App Dynamics, and Azure Application Insights using REST API. He also educated undergraduate students about big data engineering and gave them a workshop on Azure Databricks. Sannidhya has a Bachelor of Science in Computer Science from Washington State University and MAHE. He also completed a course in Mathematical Statistics and Probability from Johns Hopkins Bloomberg School of Public Health on Coursera. In addition to his technical skills, Sannidhya is a creative problem solver with excellent communication skills. He is a quick learner and highly adaptable, making him an asset to any team.